Learning Inter- and Intra-frame Representations for Non-Lambertian Photometric Stereo

In this paper, we build a two-stage Convolutional Neural Network (CNN) architecture to construct inter- and intra-frame representations based on an arbitrary number of images captured under different light directions, performing accurate normal estimation of non-Lambertian objects. We experimentally investigate numerous network design alternatives for identifying the optimal scheme to deploy inter-frame and intra-frame feature extraction modules for the photometric stereo problem. Moreover, we propose to utilize the easily obtained object mask for eliminating adverse interference from invalid background regions in intra-frame spatial convolutions, thus effectively improve the accuracy of normal estimation for surfaces made of dark materials or with cast shadows. Experimental results demonstrate that proposed masked two-stage photometric stereo CNN model (MT-PS-CNN) performs favorably against state-of-the-art photometric stereo techniques in terms of both accuracy and efficiency. In addition, the proposed method is capable of predicting accurate and rich surface normal details for non-Lambertian objects of complex geometry and performs stably given inputs captured in both sparse and dense lighting distributions.Comment: 9 pages,8 figure

Metagenomic analysis reveals taxonomic and functional diversity of microbial communities on the deteriorated wall paintings of Qinling Tomb in the Southern Tang Dynasty, China

Abstract The microbial colonization on ancient murals attracts more and more attention since the threaten by microorganisms was first reported in Lascaux, Spain. However, the biodeterioration or biodegradation of mural paintings resulted by microorganisms is not clear yet. Especially the biological function of microbial communities in different conditions remained largely unaddressed. The two mausoleums of the Southern Tang Dynasty are the largest group of emperor mausoleums during the Five Dynasties and Ten Kingdoms period in China, which are of great significance to the study of the architecture, imperial mausoleum systems and art in the Tang and Song Dynasties. To make clear the species composition and metabolic functions of different microbial communities (MID and BK), we analyzed the samples from the wall paintings in one of the two mausoleums of the Southern Tang Dynasty with metagenomics method. The result showed totally 55 phyla and 1729 genera were detected in the mural paintings. The two microbial community structure were similar with the dominance of Proteobacteria, Actinobacteria and Cyanobacteria. However, the species abundance presented a significant difference between two communities at genus level --- MID is Lysobacter, Luteimonas are predominant in MID while Sphingomonas and Streptomyces are popular in BK, which is partially attributed to the different substrate materials of murals. As a result, the two communities presented the different metabolic patterns that MID community was mainly participated in the formation of biofilm as well as the degradation of exogenous pollutants while the BK was predominantly related to the photosynthesis process and biosynthesis of secondary metabolites. Taken together, these findings indicated the effect of environmental factor on the taxonomic composition and functional diversity of the microbial populations. The installation of artificial lighting needs to be considered carefully in the future protection of cultural relics

A Novel Learning Based Non-Lambertian Photometric Stereo Method for Pixel-Level Normal Reconstruction of Polished Surfaces

High-quality reconstruction of polished surfaces is a promising yet challenging task in the industrial field. Due to its extreme reflective properties, state-of-the-art methods have not achieved a satisfying trade-off between retaining texture and removing the effects of specular outliers. In this paper, we propose a learning based pixel-level photometric stereo method to estimate the surface normal. A feature fusion convolutional neural network is used to extract the features from the normal map solved by the least square method and from the original images respectively, and combine them to regress the normal map. The proposed network outperforms the state-of-the-art methods on the DiLiGenT benchmark dataset. Meanwhile, we use the polished rail welding surface to verify the generalization of our method. To fit the complex geometry of the rails, we design a flexible photometric stereo information collection hardware with multi-angle lights and multi-view cameras, which can collect the light and shade information of the rail surface for photometric stereo. The experimental results indicate that the proposed method is able to reconstruct the normal of the polished surface at the pixel level with abundant texture information

Working memory inspired hierarchical video decomposition with transformative representations

Video decomposition is very important to extract moving foreground objects from complex backgrounds in computer vision, machine learning, and medical imaging, e.g., extracting moving contrast-filled vessels from the complex and noisy backgrounds of X-ray coronary angiography (XCA). However, the challenges caused by dynamic backgrounds, overlapping heterogeneous environments and complex noises still exist in video decomposition. To solve these problems, this study is the first to introduce a flexible visual working memory model in video decomposition tasks to provide interpretable and high-performance hierarchical deep architecture, integrating the transformative representations between sensory and control layers from the perspective of visual and cognitive neuroscience. Specifically, robust PCA unrolling networks acting as a structure-regularized sensor layer decompose XCA into sparse/low-rank structured representations to separate moving contrast-filled vessels from noisy and complex backgrounds. Then, patch recurrent convolutional LSTM networks with a backprojection module embody unstructured random representations of the control layer in working memory, recurrently projecting spatiotemporally decomposed nonlocal patches into orthogonal subspaces for heterogeneous vessel retrieval and interference suppression. This video decomposition deep architecture effectively restores the heterogeneous profiles of intensity and the geometries of moving objects against the complex background interferences. Experiments show that the proposed method significantly outperforms state-of-the-art methods in accurate moving contrast-filled vessel extraction with excellent flexibility and computational efficiency

ReDDLE-Net: Reflectance Decomposition for Directional Light Estimation

The surfaces of real objects can visually appear to be glossy, matte, or anywhere in between, but essentially, they display varying degrees of diffuse and specular reflectance. Diffuse and specular reflectance provides different clues for light estimation. However, few methods simultaneously consider the contributions of diffuse and specular reflectance for light estimation. To this end, we propose ReDDLE-Net, which performs Reflectance Decomposition for Directional Light Estimation. The primary idea is to take advantage of diffuse and specular clues and adaptively balance the contributions of estimated diffuse and specular components for light estimation. Our method achieves a superior performance advantage over state-of-the-art directional light estimation methods on the DiLiGenT benchmark. Meanwhile, the proposed ReDDLE-Net can be combined with existing calibrated photometric stereo methods to handle uncalibrated photometric stereo tasks and achieve state-of-the-art performance

Photometric-Stereo-Based Defect Detection System for Metal Parts

Automated inspection technology based on computer vision is now widely used in the manufacturing industry with high speed and accuracy. However, metal parts always appear in high gloss or shadow on the surface, resulting in the overexposure of the captured images. It is necessary to adjust the light direction and view to keep defects out of overexposure and shadow areas. However, it is too tedious to adjust the position of the light direction and view the variety of parts’ geometries. To address this problem, we design a photometric-stereo-based defect detection system (PSBDDS), which combines the photometric stereo with defect detection to eliminate the interference of highlights and shadows. Based on the PSBDDS, we introduce a photometric-stereo-based defect detection framework, which takes images captured in multiple directional lights as input and obtains the normal map through the photometric stereo model. Then, the detection model uses the normal map as input to locate and classify defects. Existing learning-based photometric stereo methods and defect detection methods have achieved good performance in their respective fields. However, photometric stereo datasets and defect detection datasets are not sufficient for training and testing photometric-stereo-based defect detection methods, thus we create a photometric stereo defect detection (PSDD) dataset using our PSBDDS to eliminate gaps between learning-based photometric stereo and defect detection methods. Furthermore, experimental results prove the effectiveness of the proposed PSBBD and PSDD dataset

ReDDLE-Net: Reflectance Decomposition for Directional Light Estimation

The surfaces of real objects can visually appear to be glossy, matte, or anywhere in between, but essentially, they display varying degrees of diffuse and specular reflectance. Diffuse and specular reflectance provides different clues for light estimation. However, few methods simultaneously consider the contributions of diffuse and specular reflectance for light estimation. To this end, we propose ReDDLE-Net, which performs Reflectance Decomposition for Directional Light Estimation. The primary idea is to take advantage of diffuse and specular clues and adaptively balance the contributions of estimated diffuse and specular components for light estimation. Our method achieves a superior performance advantage over state-of-the-art directional light estimation methods on the DiLiGenT benchmark. Meanwhile, the proposed ReDDLE-Net can be combined with existing calibrated photometric stereo methods to handle uncalibrated photometric stereo tasks and achieve state-of-the-art performance

A predictive model to estimate the pretest probability of metastasis in patients with osteosarcoma

Main portal; Vasari received his three largest architectural commissions in 1559-1562: the Uffizi in Florence, buildings for the Order of the Cavalieri di S Stefano in Pisa, which involved the remodelling of the town centre, and the dome of the Madonna dell'Umiltà in Pistoia. In 1561 he began planning the buildings in Pisa for the Order of the Cavalieri di S Stefano, founded by Cosimo, and the remodelling of the town centre. These projects, mainly concerned with rebuilding older palazzi, were intended to eradicate traces of the governmental seat of the Republic of Pisa. Work began in 1562 with the remodelling of the Palazzo dei Cavalieri, for which Vasari designed a sgraffito façade. He also designed the church of S Stefano (1565-1569) and its interior decoration; later he painted its altarpiece. The Canonica and the Palazzo dell'Orologio, planned and begun by Vasari, were completed in the 17th century. Source: Grove Art Online; http://www.groveart.com/ (accessed 2/3/2008